Design from Policies: Conservative Test-Time Adaptation for Offline Policy Optimization

In this work, we decouple the iterative bi-level offline RL (value estimation and policy extraction) from the offline training phase, forming a non-iterative bi-level paradigm and avoiding the iterative error propagation over two levels. Specifically, this non-iterative paradigm allows us to conduct inner-level optimization (value estimation) in training, while performing outer-level optimization (policy extraction) in testing. Naturally, such a paradigm raises three core questions that are not fully answered by prior non-iterative offline RL counterparts like reward-conditioned policy: (q1) What information should we transfer from the inner-level to the outer-level? (q2) What should we pay attention to when exploiting the transferred information for safe/confident outer-level optimization? (q3) What are the benefits of concurrently conducting outer-level optimization during testing? Motivated by model-based optimization (MBO), we propose DROP (design from policies), which fully answers the above questions. Specifically, in the inner-level, DROP decomposes offline data into multiple subsets, and learns an MBO score model (a1). To keep safe exploitation to the score model in the outer-level, we explicitly learn a behavior embedding and introduce a conservative regularization (a2). During testing, we show that DROP permits deployment adaptation, enabling an adaptive inference across states (a3). Empirically, we evaluate DROP on various tasks, showing that DROP gains comparable or better performance compared to prior methods.Comment: NeurIPS 202

Chain of Thought Prompt Tuning in Vision Language Models

Language-Image Pre-training has demonstrated promising results on zero-shot and few-shot downstream tasks by prompting visual models with natural language prompts. However, most recent studies only use a single prompt for tuning, neglecting the inherent step-to-step cognitive reasoning process that humans conduct in complex task settings, for example, when processing images from unfamiliar domains. Chain of Thought is a simple and effective approximation to human reasoning process and has been proven useful for natural language processing (NLP) tasks. Based on this cognitive intuition, we believe that conducting effective reasoning is also an important problem in visual tasks, and a chain of thought could be a solution to this problem. In this work, we propose a novel chain of thought prompt tuning for vision-language modeling. Extensive experiments show that our method not only generalizes better in image classification tasks, has greater transferability beyond a single dataset, and has stronger domain generalization performance, but also performs much better in imagetext retrieval and visual question answering, which require more reasoning capabilities. We are the first to successfully adapt chain-of-thought prompting that combines visual and textual embeddings. We will release our code

Targeted Poverty Alleviation and Households’ Livelihood Strategy in a Relation-Based Society: Evidence from Northeast China

Abstract: Although China is experiencing a transition from a relation-based society to a rule-based society, relationships among acquaintances still play an important role in resource allocation, such as the allocation of policy resources. This is particularly true in rural China, where targeted poverty alleviation is prevalent and a relation-based social structure still dominates. However, it is still unknown how relationships affect the livelihood strategy of households in rural China and how poverty alleviation policies plays a role between them. Therefore, this paper embeds poverty alleviation into the relation-based society and explores how households respond to the policy in this specific context. Using grounded theory research method and the sustainable livelihoods approach (SLA) framework, this paper contains in-depth interviews and field observations from three povertystricken villages in Northeast China. The results show that relationships have a significant impact on the households’ livelihood strategy. In other words, the households’ livelihood strategy is embedded in the relation-based society. The types of relationships induce households to choose maintained or developmental type livelihood strategies, while relationships influence how the poverty alleviation policies affect the livelihood strategy. This study is not only an extension of the SLA in the research context, but also provides a significant perspective for enriching the long-term mechanism of targeted poverty alleviation by building a theoretical model of the relationships between a relation-based society, targeted poverty alleviation and the livelihood strategies of households.publishedVersio

Experimental Exploration of Influence of Recycled Polymer Components on Rutting Resistance and Fatigue Behavior of Asphalt Mixtures

Rutting and fatigue of asphalt pavements, as two important distresses, are significantly influenced by the properties of binders. This study aimed to improve the resistance of asphalt mixtures to permanent deformation and fatigue using two recycled waste-polymer components in recycled crumb rubber (CR) and polyethylene (PE). The assessed pavement properties of the modified asphalt mixtures were characterized by wheel tracking, uniaxial penetration, and four-point bending (4PB) tests. The wheel tracking test indicated that the integrated modification technique, by functionally incorporating PE and CR, enhanced the dynamic stability of the asphalt mixtures and that PE dosage was a key variable. From the uniaxial penetration test, it was revealed that the shear strength of the asphalt mixtures at high temperature could be improved by the integrated modification method, indicating the method’s potential to reduce the flow rutting of asphalt pavements. Meanwhile, both the CR and PE were shown to increase the cohesive behavior of the asphalt mixtures, with the friction angle value sensitive to PE dosage. The addition of PE reduced the fatigue life of the asphalt mixtures; the CR improved the PE-modified mixtures’ fatigue resistance. The findings from this study will be beneficial in developing sustainable and durable asphalt pavements, tailoring the reuse of different types of polymer wastes in asphalt pavements, and minimizing waste disposal at landfills

RSG: Fast Learning Adaptive Skills for Quadruped Robots by Skill Graph

Developing robotic intelligent systems that can adapt quickly to unseen wild situations is one of the critical challenges in pursuing autonomous robotics. Although some impressive progress has been made in walking stability and skill learning in the field of legged robots, their ability to fast adaptation is still inferior to that of animals in nature. Animals are born with massive skills needed to survive, and can quickly acquire new ones, by composing fundamental skills with limited experience. Inspired by this, we propose a novel framework, named Robot Skill Graph (RSG) for organizing massive fundamental skills of robots and dexterously reusing them for fast adaptation. Bearing a structure similar to the Knowledge Graph (KG), RSG is composed of massive dynamic behavioral skills instead of static knowledge in KG and enables discovering implicit relations that exist in be-tween of learning context and acquired skills of robots, serving as a starting point for understanding subtle patterns existing in robots' skill learning. Extensive experimental results demonstrate that RSG can provide rational skill inference upon new tasks and environments and enable quadruped robots to adapt to new scenarios and learn new skills rapidly

Natural Language Embedded Programs for Hybrid Language Symbolic Reasoning

How can we perform computations over natural language representations to solve tasks that require symbolic and numeric reasoning? We propose natural language embedded programs (NLEP) as a unifying framework for addressing math/symbolic reasoning, natural language understanding, and instruction following tasks. Our approach prompts a language model to generate full Python programs that define functions over data structures which contain natural language representations of structured knowledge. A Python interpreter then executes the generated code and prints the output. Despite using a task-general prompt, we find that this approach can improve upon strong baselines across a range of different tasks including math and symbolic reasoning, text classification, question answering, and instruction following. We further find the generated programs are often interpretable and enable post-hoc verification of the intermediate reasoning steps

Variation in VEGFA and risk of cardiovascular disease in the UK Biobank

BackgroundCardiovascular disease (CVD) is an escalating global health crisis, contributing significantly to worldwide mortality and morbidity. Dyslipidemia stands as a critical risk factor for CVD. Vascular endothelial growth factor A (VEGFA) is pivotal in angiogenesis and represents a clinical target for CVD intervention. However, the impact of genetic modulation of VEGFA on lipid levels and the subsequent risk of cardiovascular events remains unclear.MethodsWe used LDpred2 to calculate genetic scores for lipid levels based on VEGFA variation, serving as instrumental variables to simulate the effect of VEGFA inhibitors. We then assessed the associations between genetic risk for lipid levels and CVD risk by conducting One-sample Mendelian randomization.ResultsOur results indicated that low-density lipoprotein cholesterol [LDL-C; odds ratio (OR) = 1.09, 95% CI: 1.06–1.11], remnant cholesterol (RC; OR = 1.24, 95% CI: 1.13–1.36), and triglycerides (TG; OR = 1.14, 95% CI: 1.07–1.22) were positively associated with the incidence of CVD. In contrast, high-density lipoprotein cholesterol (HDL-C) was inversely associated with the incidence of CVD (OR = 0.80, 95% CI: 0.76–0.86). When considering the genetic score for LDL-C constructed based on VEGFA, the group with a high genetic score demonstrated an elevated CVD risk (OR = 1.11, 95% CI: 1.04–1.19) compared to those with a low genetic score. Notably, One-sample Mendelian randomization results provided evidence of a causal relationship between LDL-C and CVD (p = 8.4×10−3) when using genetic variation in VEGFA as an instrumental variable.ConclusionsGenetic variation mimicking the effect of VEGFA inhibition, which lowers LDL-C levels, was causally associated with a reduced risk of cardiovascular events. These findings offer insight into the potential therapeutic relevance of modulating VEGFA-mediated lipid changes in the prevention and management of CVD